Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add two log output modes, stdout-full and stdout-new-only #3439

Closed
wants to merge 9 commits into from

Conversation

rafaeltab
Copy link
Contributor

Add two log output modes, stdout-full and stdout-new-only

The new stdout-full mode will not prefix the output generated by the commands run by Turborepo. It will still prefix turbos messages, such as cache hit, cache miss, and cache bypass. This allows the user to identify what was and wasn't cached while preserving the original output.

The new stdout-new-only mode combines the behavior of new-only and stdout-full by only logging non-cached tasks, without prefixing the output.

stdout-full

Command: turbo run build --output-logs=stdout-full --filter=util --force
Output:

\xe2\x80\xa2 Packages in scope: util
\xe2\x80\xa2 Running build in 1 packages
\xe2\x80\xa2 Remote caching disabled
util:build: cache bypass, force executing 6dec18f9f767112f
  
> build 
> echo 'building'
  
building
  
 Tasks:    1 successful, 1 total
Cached:    0 cached, 1 total
  Time:    354ms

stdout-new-only

Command: turbo run build --output-logs=stdout-new-only --filter=util
Output:

\xe2\x80\xa2 Packages in scope: util
\xe2\x80\xa2 Running build in 1 packages
\xe2\x80\xa2 Remote caching disabled
util:build: cache hit, suppressing output 6dec18f9f767112f

 Tasks:    1 successful, 1 total
Cached:    1 cached, 1 total
  Time:    5ms >>> FULL TURBO

Command: turbo run build --output-logs=stdout-new-only --filter=util --force
Output:

\xe2\x80\xa2 Packages in scope: util
\xe2\x80\xa2 Running build in 1 packages
\xe2\x80\xa2 Remote caching disabled
util:build: cache bypass, force executing 6dec18f9f767112f
  
> build
> echo 'building'
  
building
  
 Tasks:    1 successful, 1 total
Cached:    0 cached, 1 total
  Time:    243ms

Side notes

I also added some integration tests for the output-logs option

Closes #901

@rafaeltab rafaeltab requested a review from a team as a code owner January 23, 2023 20:48
@vercel
Copy link

vercel bot commented Jan 23, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated
examples-designsystem-docs 🔄 Building (Inspect) Jan 23, 2023 at 8:54PM (UTC)
examples-tailwind-web ✅ Ready (Inspect) Visit Preview 💬 Add your feedback Jan 23, 2023 at 8:54PM (UTC)
6 Ignored Deployments
Name Status Preview Comments Updated
examples-basic-web ⬜️ Ignored (Inspect) Jan 23, 2023 at 8:54PM (UTC)
examples-cra-web ⬜️ Ignored (Inspect) Jan 23, 2023 at 8:54PM (UTC)
examples-native-web ⬜️ Ignored (Inspect) Jan 23, 2023 at 8:54PM (UTC)
examples-nonmonorepo ⬜️ Ignored (Inspect) Jan 23, 2023 at 8:54PM (UTC)
examples-svelte-web ⬜️ Ignored (Inspect) Jan 23, 2023 at 8:54PM (UTC)
turbo-vite-web ⬜️ Ignored (Inspect) Jan 23, 2023 at 8:54PM (UTC)

@vercel
Copy link

vercel bot commented Jan 23, 2023

@rafaeltab is attempting to deploy a commit to the Vercel Team on Vercel.

A member of the Team first needs to authorize it.

Copy link
Contributor

@mehulkar mehulkar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution @rafaeltab! The implementation and tests look great, but I don't love the user-facing API.

  1. "stdout" is inaccurate, as it will stream stderr also
  2. Adding stdout-* options means that we need 2n options for each "filter view" of logs.

What do you think instead about adding a new flag --pretty and making it on by default? Then, --output-logs=stdout-full would be the same as --output=logs=full, etc. Some advantages of this:

  • we don't need to capture what turbo means by pretty output, we can do prefixes, colors, or something else in the future
  • following git log --pretty, in the future we could enable something like --pretty='%some-micro-syntax-for-fine-grain-control%'

I'm not sold on this idea, since adding a new flag adds some overhead, but I hope this gives us something to talk about!

@rafaeltab
Copy link
Contributor Author

I agree, the name stdout doesn't really convey the idea that prefixes are removed either. A pretty flag could cover this, however, we do not want to make this too complex either. Perhaps some baked-in pretty configurations, with the option to go custom would be best. Some ideas that come to mind are

turbo run build & turbo run build --pretty & turbo run build --pretty=normal: Current output

turbo run build --pretty=none: Remove prefixes, basically the current result from stdout-full

turbo run build --pretty=no-color: This would remove the color from the prefixes we have currently, it should not affect the logs themselves.

turbo run build --pretty=folder-name: This would log the folder name of the workspace instead of the package name

turbo run build --pretty="%Cred%pn%Creset: %O": Custom format, print the package name in red
%Cred to get color red, %pn for package name, %Creset to reset the color, %o for output

If we were to go for this option this would require a rewrite for a large portion of the logging system.
And of course more investigation, I believe that would fall out of the scope for this PR though, so how about implementing --pretty=none, --pretty, and --pretty=normal for now? That way the team can have some time to think this idea over properly, and probably create an internal or public RFC.

This would also allow the current stdout behavior to be combined with errors-only, which was left out in this PR.

@mehulkar
Copy link
Contributor

how about implementing --pretty=none, --pretty, and --pretty=normal for now?

I think we can start with only true/false (defaulting to true). My suggestion to expand the flag was to demonstrate how --pretty would be more open for expansion, rather than to ask for this! Definitely don't go down the rabbit hole rewriting logging! :)

cc @gsoltis / @vercel/turbo-oss to help make a decision/bikeshed here as well.

One more thing to point out is that without prefixing, logs for tasks running in parallel will be mixed up. The prefixes are the only thing that let you differentiate between them right now. That may be a reason to add --pretty=no-color (which would leave prefixes) in addition to true/false options. Or it could just be documented if we decide to tackle #219 at some point.

@rafaeltab
Copy link
Contributor Author

I was absolutely not planning to rewrite logging just yet, the 3 possible options I could implement all represent either the no prefix logging or the normal logging, so the current implementation could almost completely work for them. Then when you guys decide you want to go all the way we won't have any backwards compatibility issues.

If you still prefer a simple boolean, that is also still an option. Would that be --pretty and --pretty false or --no-pretty or both.

@rafaeltab
Copy link
Contributor Author

The logs will indeed be mixed up, and --grouped would solve this, I am probably going to try picking that up once this is finished as well. It also seems to be the case that people don't want to remove the prefix because they think it is ugly, but because it screws up some CI mechanisms (such as set-output for github).

@gsoltis
Copy link
Contributor

gsoltis commented Jan 27, 2023

How about a separate boolean flag like --raw-output or --no-prefix-logs or something like that? That way it's separate from which logs to print. Would that cover what we need?

@rafaeltab
Copy link
Contributor Author

@gsoltis that would absolutely cover what we need. I think it's more a question of what do we want to add in the future when it comes to logging, and what do we think looks and feels better.

I believe --no-prefix, --no-prefix-logs, --raw-output & --pretty=false all cover what we currently need. The --pretty option allows for more future flexibility, and --no-prefix/--no-prefix-logs reflects the current behavior in the best way.

Do we want more flexibility, or de we want more clarity?

@gsoltis
Copy link
Contributor

gsoltis commented Jan 28, 2023

I see. Let's go with --raw-logs. I think that communicates what we're doing and optimizes for clarity now. If we need more flexibility in the future, we can either deprecate or mark as incompatible with whatever flags we end up needing then.

@zomars
Copy link

zomars commented Feb 20, 2023

Can we take over this PR?

@rafaeltab
Copy link
Contributor Author

I have been a bit busy recently (I will be moving out of my parents' home to live with a friend of mine), I will continue development for this PR this week.

@mehulkar
Copy link
Contributor

mehulkar commented Feb 21, 2023

Hey @rafaeltab thanks for your work on this. We talked internally and landed on --log-prefix. I’ve started that work in #3851 and it’s my top priority tomorrow. We expect it to land in a 1.8.x patch release! Feel free to leave any feedback there! Also happy to share more of our thinking around the naming choice, etc.

@rafaeltab
Copy link
Contributor Author

Alright, it's understandable since I hadn't worked on it for a little while. The naming choice seems straightforward as well. I'll close this PR then. I'll probably end up picking up grouped output in the near future.

@rafaeltab rafaeltab closed this Feb 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: logging Improvements to logging
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Do not prefix log output
5 participants